Cooperation in noisy case: prisoner's dilemma game on two types of regular random graphs 
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We have studied an evolutionary prisoner's dilemma game with players located on two types of random 
regular graphs with a degree of 4. The analysis is focused on the effects of payoffs and noise (temperature) on 
the maintenance of cooperation. When varying the noise level and/or the highest payoff, the system exhibits 
a second order phase transition from a mixed state of cooperators and defectors to an absorbing state where 
only defectors remain alive. For the random regular graph (and Bethe lattice) the behavior of the system is 
similar to those found previously on the square lattice with nearest neighbor interactions, although the measure 
of cooperation is enhanced by the absence of loops in the connectivity structure. For low noises the optimal 
connectivity structure is built up from randomly connected triangles. 
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One of the central questions in evolutionary game theory 
is to find necessary conditions and mechanisms which result 
in cooperation among selfish individuals. Naturally, several 
mechanisms have already been explored such as kin selection 
retaliating behavior |2J, voluntary participation |3| or de- 
velopment of reputation 0] . 

A widely studied toy model, which illustrates the conflict 
between cooperation and selfish behavior is the prisoner's 
dilemma game (PDG). In the (two-player and one-shot) PDG 
||5l|^, the players simultaneously make decision whether co- 
operate or defect to maximize their individual payoff. The 
dilemma is based on the fact that defection brings higher in- 
come independently of the other player's decision. But if both 
players defect they receive significantly lower payoff than in 
the case of mutual cooperation. 

The introduction of short-range interaction between the 
spatially distributed players can explain the formation of co- 
operation for the iterated evolutionary PDGs |7, 8], even for 
the case when players can follow one of the two simplest 
strategies as "always cooperate" (C) and "always defect" (D). 
In multi-agent evolutionary PDGs, the players gain their in- 
come from games with their neighbors. According to the 
Darwinian selection principle, the less successful strategies 
are replaced by more successful strategies adopted from their 
neighborhood 1 9, 10|. 

Lots of studies followed the pioneering work of Nowak et 
at. Q 01 in the field of evolutionary PDGs analyzing many 
types of connectivity structures (lattices 1 11, 12, 13], diluted 
lattices |8], social networks |14, 15, 16, 17], hierarchical 
graphs ] 18], scale-free networks ] 19], preferential selection of 
a neighbor 120]) to clarify the possible role of the topology. In 
Ref. ]21], self organizing networks controlled by proper dy- 
namics were applied to find the most advantageous structure 
for cooperation. Our previous study ll22il reveals the impor- 
tance of clique percolation fl?,^!^. 

In the present paper, as a continuation of our previous work 
i23l . we explore the effect of noise parameter on the PDG by 
extending the connectivity structures to random graphs. Our 
observations suggest that the effects of noise (temperature) on 
the stationary concentration of cooperators is greatly affected 
by the underlying structure. 



To investigate the influence of the noise and connectiv- 
ity structures, we have considered an evolutionary PDG with 
players located on the sites of a regular graph with a connec- 
tivity of four (z = 4). The restriction to regular random graph 
serves to make the comparison easier and to avoid undesired 
effects due to the various degrees of nodes (different sizes of 
neighborhoods) mHllllii. 

The players can follow one of the above mentioned two 
strategies whose distribution is described by the formalism of 
the two-state Potts model: the possible state of the site x is 
Sx = C or D (cooperator or defector). The strategy adoption 
mechanism ] 22] is based on the rescaled version of the payoff 
matrix introduced by Nowak et at. |0]: 
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1 < 6 < 2 



(1) 



During the evolutionary process the randomly chosen player 
at site X can adopt one of the (randomly chosen) co-player's 
(at site y) strategy with a probability depending on the payoff 
difference [Ux — Uy) 
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where K characterizes the magnitude of noise involving many 
different effects (fluctuations in payoffs, errors in decision, in- 
dividual trials, etc.) IIiIEtIi. 

The present analysis is focused on two connectivity struc- 
tures displayed in Fig. Q Henceforth, we will refer to the 
structures as RRGl and RRG2, respectively. The RRGl 
random regular graph is the simulated version of the Bethe- 
lattice, because RRGl is locally similar to a tree in the large 
size limit (when the number of sites goes to infinity). For finite 
number N of sites, however, RRGl has loops. This influence 
seems to be negligible when choosing sufficiently large sys- 
tems. To avoid confusion and for better visualization. Fig. [J 
represents an "ideal" (free of loops) part of RRGl. 

RRG2 is a random regular graph of triangles with three 
neighboring triangles. The triangles of RRG2 are the nodes 
of the underlying random regular graph (with z = 3), and 
between these nodes, the bonds are the common sites of the 
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FIG. 1 : Two types of random regular graphs on which an evolution- 
ary Prisoner's Dilemma game is studied. 



overlapping triangles. Locally, it resembles to the Kagome 
lattice, but - as will be seen later - it has some characteristics 
of a random regular graph, too. Despite the similar coordina- 
tion number {z — 4) the topology of the two random networks 
differs significantly. In the limit N oo, the concentration 
of triangles (three sites cliques) vanishes in RRGl, i.e., this 
structure has a clustering coefficient C = 0, whereas the over- 
lapping triangles percolate on RRG2, and the clustering coef- 
ficient is C = 1/3. 

Since the classical mean-field theory is insensible to the 
topology, the equation of motion for the concentration of co- 
operators is identical with the equation of the model on lattices 
presented in our previous work |22]. The solution of differen- 
tial equation suggests that cooperators die out and defection 
is the only successful strategy for arbitrary values of K and 
b > 1. 

As it was mentioned in our previous paper i22il the neces- 
sary condition for cooperators to survive is the possibility to 
form clusters where cooperators can assist each other. This 
may happen on lattices in higher dimensions at certain range 
of parameter b. The present paper surveys the areas of the b-K 
parameter plain where cooperation can survive. 
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FIG. 2: Monte Carlo results for the concentration of cooperators vs. 
b for three different noise values: K — 0.1 (pluses), 0.3 (diamonds), 
and 1.2 (squares) on RRGl. 

Figure 12] shows the stationary concentration p of cooper- 
ators on RRGl as a function of b for different temperature 



values. This plot indicates that the variation of p on RRGl is 
qualitatively similar to those found on the square lattice 
These numerical data are obtained from Monte Carlo (MC) 
simulations performed on large systems [the size varied from 
N = 4 • 10"* to = 4-10''], and the stationary values are 
determined by averaging over a sampling time ts [varied from 
ts = 10^ to 10^ Monte Carlo steps per sites]. The larger 
N and tg are used in the close vicinity of the extinction of 
cooperators because, the variance diverges approximately as 
X oc p^^ for small concentration and low K. Further difficul- 
ties were caused by the size dependence at low K values. The 
reason of this phenomenon can be related to the existence of 
small loops whose effect will be discussed later on. 

The transition from a fluctuating (C + D) phase into the 
absorbing state frequently belongs to the directed percola- 
tion universality class. The critical exponent characterizing 
the power law decrease of concentration were reproduced 
when the evolutionary PDG was simulated on different two- 
dimensional lattices 1 13, 22]. As expected, the change of host 
lattice to random graph results in a mean-field type behavior in 
p{K) as demonstrated in Fig.|2] The stationary concentration 
of cooperators is independent of the initial state and decreases 
linearly with 6cr — b. As 6 exceeds a threshold value (bcr), 
cooperation cannot be maintained, the evolution always ends 
in the homogeneous D state. 




FIG. 3: Critical value of 6 vs. K on RRGl. Symbols come from 
Monte Carlo simulations, the solid, dashed, dotted, and dashed- 
dotted lines represent the predictions of dynamical cluster approx- 
imation for 2-, 5-, 8-, and ll-site clusters shown at the top. 

We have determined the critical values of b for many differ- 
ent noises and the results are illustrated in Fig.|3] The bcr{K) 
curve has a maximum at X « 0.37, and goes to 1 if X goes 
either to zero or to infinity. This figure can be considered as 
a phase diagram because the solid line connecting the Monte 
Carlo data separates the area where cooperators and defectors 
can coexist from the pure defector area. 

The possibility of a similar stochastic resonance was re- 
ported by Traulsen et al. |28] who considered another evo- 
lutionary rule based on the application of the "win-stay-lose- 
shift" strategies. The appearance of stochastic resonance was 
directly demonstrated on the square lattice by Perc in a similar 
PDG with adding random perturbations to the payoffs 1 291 . 
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The analytical reproduction of this behavior proved to be 
a very time-consuming task. In contrary to the Monte Carlo 
results, as detailed in the previous paper i22ll . the traditional 
mean-field approximation suggests a sudden change between 
the homogeneous D and C states at bl^^'^K) = 1. The pan- 
approximation is capable to describe the coexistence of the C 
and D strategies but it gives a rough estimation for critical b 
values, especially in the zero temperature limit. To eliminate 
these discrepancies we have to extend the dynamical cluster 
approximations (considered as generalized mean-field meth- 
ods) on the Bethe lattice |30]. When applying these tech- 
niques we derive equation of motion for all possible configu- 
rational probabilities on large clusters and search numerically 
for the stationary concentrations by integrating the equations 
of motion with respect to time (further details on these meth- 
ods will be given elsewhere). Figure |3] shows that both the 
five- and eight-site approximations predict incorrect results in 
the zero temperature limit although their predictions become 
more and more accurate in the high temperature region. Be- 
sides it, one can observe relevant improvement when compar- 
ing the results of the five- and eight-site approximations at the 
limit K ^ 0. This fact inspired us to extend this method 
for the level of 11 -site approximations. As shown in Fig.|3l 
this level is already capable of describing the disappearance 
of cooperation as K goes to 0. The above series of results 
emphasizes the importance of long-range correlations and/or 
the absence of loops in the connectivity structures for the limit 
K 0. 

Basically different behavior is found on the RRG2 struc- 
ture as demonstrated in Fig. |4] It is conjectured previously 
that the function bcr{K) decreases monotonously to 1 if ii' 
is increased for those connectivity structures where the over- 
lapping triangles spans the whole system. The present data 
support this conjecture. 
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FIG. 4: Phase diagram on RRG2. Symbols denote the MC data. The 
dashed and dotted lines illustrate the phase boundary between the D 
and (C + D) phases predicted by the 3- and 5-site approximation on 
the clusters shown at the top. 

In Fig.|4]we compare the MC data with the predictions of 
three- and five-site approximations. It is worth mentioning 



that qualitatively similar results are obtained when the con- 
nectivity structure is given by the Kagome lattice ||22,1 . The 
RRG2 and Kagome lattice are locally similar: the overlap- 
ping triangles have only one common site. This is the main 
reason why the three- and five-site approximations predict the 
same results on the RRG2 and Kagome lattice. In fact, the 
structure of RRG2 fits well to the conditions of the five-site 
approximations overestimating both p and bcr for the Kagome 
lattice. At the same time, the results of this approach agrees 
very well with the MC data on RRG2 (see Fig.|3. We have to 
emphasize that this is the connectivity structure providing the 
highest measure of cooperation among regular structures (if 
2 = 4) for low K values (for a comparison see Fig.|5}. At the 
first glance it is a surprising result because the introduction of 
spatial connectivity structures was motivated by the possibil- 
ity of the formation of C colonies t^JJ. 
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FIG. 5: Phase boundaries between the D and (C + D) phases 
on RRGl (pluses), RRG2 (double-triangle symbols), square lattice 
(squares), and Kagome lattice (triangles). 

The above results have helped us to deduce a simple expla- 
nation justifying the importance of the one-site overlapping 
triangles in the connectivity structures. Let us assume that 
one of the triangles is occupied by cooperators in the sea of 
defectors. Within this triangle, the cooperator's income is 2, 
the neighboring defectors receive 6, and all the other _Ds get 
nothing. In this situation the most probable evolutionary pro- 
cess is that one of the neighboring defectors adopts the strat- 
egy of the more successful cooperators (for low K). The state 
of this new cooperator is not stable and it can be switched 
into defector again within a short time. During the lifetime of 
the new cooperator, however, the other neighboring defector 
adopts the C strategy very probably from the cooperator in 
the original triangle (if 6 < 3 /2), and they will form a neigh- 
boring (stable) triplet of cooperators. The iteration of these 
processes yields a growing tree of cooperator triplets. The 
growing process is stopped at the sites which separate two 
branches of the growing tree, because the corresponding de- 
fector(s) can exploit two or more cooperators simultaneously. 
The blocking events occur frequently for the spatial structures 
and are excluded for tree-like structures. Thus, the absence of 
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loops (formed by the one-site overlapping triangles) sustain 
the spreading of cooperation in RRG2. This picture is valid 
if only one tree of cooperator triplets exists in the initial state. 
For random initial state, however, the system will have many 
(tree of) cooperator triplets which will check each other in the 
growing process at the sites where their branches are separated 
even on the RRG2 structure. In the final stationary state the 
spreading and blocking effects are balanced in a way taking 
the noise into account. Evidently, the noisy effects can break 
up the triplets of cooperators. As a result, one can observe 
monotonous decrease in the function p{K) when increasing 
K for any b < bcr- 

Disregarding the triangles, the presence of loops in the con- 
nectivity structure reduces the measure of cooperation for high 
noises too. Figure [S] clearly demonstrates that RRG2 is more 
advantageous than the Kagome lattice in the maintenance of 
cooperation. Similar conclusion can be deduced when we 
compare the results obtained on the RRGl and square lattice 
(see Fig. |5}- In these cases the stable triplets of cooperators 
cannot be observed because of the absence of triangles in the 
connectivity structure. For both structures the maintenance of 
cooperation is supported by the noisy events resembling to the 
stochastic resonance |29|. For finite noises the occasional (ir- 
rational) adoption of the neighboring C strategy increases the 
tree of cooperators, and this effect is weakened by the pres- 
ence of loops as described above. 

It is more surprising that the disadvantageous presence of 
loops (with length longer than 3) is so relevant that RRGl 
becomes the most efficient structure to sustain the coopera- 



tion if K exceeds a threshold value (K > Ktu — 0.4) for 
z = 4. In other words, in the maintenance of cooperation the 
advantageous effect of the quenched (regular) neighborhood 
is weakened by the spatial characteristics of the connectivity 
structure. 

To summarize, we have systematically studied the effect of 
noise K and temptation h to choose defection on the mea- 
sure of cooperation in an evolutionary PD game for two types 
of random regular connectivity structures with z — A. For 
this purpose we have determined the critical value bcr{K) of 
temptation until the cooperators can remain alive. For suf- 
ficiently high noise levels the comparison of different con- 
nectivity structures indicates that the highest bcr{K) can be 
achieved by minimizing the number of loops in the connectiv- 
ity structure. On the contrary, at low noise levels, the preferred 
structure is built up randomly from overlapping triangles in 
a way that the overlapping triangles have only one common 
site. Evidently, the analysis becomes more complicated for 
those connectivity structures which involve variation in de- 
gree, z > 4, and more complex connections at the overlapping 
triangles. 
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